Python Job: Python Data Engineer

Streetbees is a market intelligence platform that gathers in-the-moment qualitative behavioural data at a quantitative scale directly from consumers, in their own words.

Unilever, Mars, PepsiCo, Sony Pictures, Carlsberg, Ikea, Vodafone, L’Oreal and other market leading brands use Streetbees consumer data insights to unlock growth.

Streetbees platform is powered by data from 4.5 million bees, across 190 countries and 2.5 billion data points.

An opportunity exists for a Python Data Engineer to join our team, particularly those who find excitement in a 10x scaling of the platform.

This is a fully hands on position. Experience with Python, Spark, SQL and CI/CD development practices will be essential while experience with AWS, CircleCI (or other automated build platforms), Terraform and Datadog are nice to have.

Role Responsibilities

• Developing and maintaining data pipelines for our Data Lake, Data Warehouse, ML training loops and data operations workflows

• Working closely with the Data Operations and BI teams to support their activities with data transformations and new data sources

• Playing an integral role in implementing our Data Architecture Strategy

• Hands-on design, development and support of applications and infrastructure in collaboration with the rest of the platform engineering team. Active pull request participation, branch management, code reviews etc

• Creating and managing infrastructure as code using Terraform and Atlantis

• Investigating bugs and performance issues and providing support to the product engineering teams

• Working closely with developers and product managers to ensure that data engineering requirements are implemented correctly and efficiently

• Being an active part of the group of engineers reviewing and making key tech design decisions, and coordinating work and operations on our platform architecture

• Sharing knowledge within the engineering team and data analysts using presentations, demos and pair programming

• Working on production deployment of Machine Learning AI solutions (DataOps for machine learning is a new area for the industry, with little current conventional best practice, Streetbees expertise in ML creates a real opportunity to do leading-edge DataOps work productionising that)

• Keeping up to date with the latest evolutions in engineering best practice and technologies

Requirements

• Experience with Python including pyspark, pandas, numpy is essential

• Experience with Spark, AWS Glue, Athena, Delta Lake, Redshift

• Experience managing relational databases (we use Aurora Postgres)

• Experienced with the AWS cloud platform

• Experience with containers and scheduling tools (we use Docker on ECS)

• Experience managing infrastructure as code with Terraform

• Experience working with CI/CD pipeline tools such as CircleCI, Jenkins etc

Benefits

• A 24-day paid holiday allowance with the option to roll over up to 3 days per year

• Competitive salary & share options

• Option to choose between a Mac or a PC (we will send this out via courier a few days before you join!)

• Flexible working hours

• A pension scheme - our new pension scheme means that Streetbees will start paying a percentage of your salary towards your pension.

• The Streetbees Lifestyle Committee - any employee will have the opportunity to plan and manage a variety of Streetbees funded socials and activities for the team. Socials and day outs can involve anything from rock climbing or bowling to company meals or pub crawls.

• Friday Lunch - If you’re in the office on a Friday, lunch is on us!

• Office Food - plenty of awesome drinks and snacks on hand in our office, all free!

• Friday Catch-ups - Beers, wine, soft drinks and snacks provided as we celebrate the week's successes and welcome newcomers.

• Employee Assistance Programme - provides a complete support network for all of our employees that offers expert advice and compassionate guidance 24/7, covering a wide range of issues (not just work-related).

• Paid volunteering days - one day per quarter with a registered charity of your choice.

• Audible subscriptions - the company is more than happy to pay for audio-books subscriptions of your choice on Amazon as we encourage wider and continued

• learning among our team.

• Cycle to Work Scheme (reach out to [email protected] to find out more about this!)

• Yoga - We have Yoga every Tuesday at 6:30 pm, both in the office and live on Zoom so you can join in anywhere you like.